Information processing apparatus, information processing system, and information processing method

ABSTRACT

There is provided an information processing apparatus including: a process execution unit configured to execute a process relating to a user&#39;s gesture recognized on a basis of information from a sensor. The process execution unit determines, during a period after the gesture has been recognized, whether or not there is an input of operation information based on a user&#39;s operation, and the process execution unit refrains from executing the process relating to the recognized gesture when there is an input of the operation information during the period.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/584,411, filed May 2, 2017, which claims the benefit of JapanesePriority Patent Application JP 2016-099393, filed May 18, 2016, theentire contents of each are incorporated herein by reference.

BACKGROUND

The present disclosure relates to an information processing apparatus,an information processing system, and an information processing method.

In recent years, users can operate an information processing apparatusby various operation methods. For example, an information processingapparatus has been developed which detects a user's gesture to allow auser's operation.

JP 2010-191826A discloses an information processing apparatus whichallows a user's operation on the basis of the user's gesture asdescribed above. The information processing apparatus disclosed in JP2010-191826A includes an imaging device, and the imaging device detectsthe user's gesture.

SUMMARY

There may be a case where the user performs an erroneous operation whenperforming an operation on the information processing apparatus. Thesame goes for a case where the user performs a gesture operation. It istherefore desirable to provide a mechanism for allowing the user tocancel, when the user has performed a gesture operation, the operationbased on the gesture.

According to an embodiment of the present disclosure, there is providedan information processing apparatus including: a process execution unitconfigured to execute a process relating to a user's gesture recognizedon a basis of information from a sensor. The process execution unitdetermines, during a period after the gesture has been recognized,whether or not there is an input of operation information based on auser's operation, and the process execution unit refrains from executingthe process relating to the recognized gesture when there is an input ofthe operation information during the period.

According to an embodiment of the present disclosure, there is providedan information processing system including: a process execution unitconfigured to execute a process relating to a user's gesture recognizedon a basis of information from a sensor. The process execution unitdetermines, during a period after the gesture has been recognized,whether or not there is an input of operation information based on auser's operation, and the process execution unit refrains from executingthe process relating to the recognized gesture when there is an input ofthe operation information during the period.

According to an embodiment of the present disclosure, there is providedan information processing method including: executing a process relatingto a user's gesture recognized on a basis of information from a sensor;determining, during a period after the gesture has been recognized,whether or not there is an input of operation information based on auser's operation; and refraining from executing the process relating tothe recognized gesture when there is an input of the operationinformation during the period.

According to the present disclosure described above, when the user hasperformed an erroneous operation on the basis of a gesture, the user cancancel the erroneous operation.

Note that the effects described above are not necessarily limitative.With or in the place of the above effects, there may be achieved any oneof the effects described in this specification or other effects that maybe grasped from this specification.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an exemplary information processingsystem of an embodiment of the present disclosure;

FIG. 2 is a diagram illustrating an exemplary appearance of a small-sizeterminal of an embodiment of the present disclosure;

FIG. 3 is a block diagram illustrating an exemplary configuration of aninformation processing system of an embodiment of the presentdisclosure;

FIG. 4 is a flowchart illustrating an exemplary process in aninformation processing system of an embodiment of the presentdisclosure;

FIG. 5 is a flowchart illustrating an exemplary process in aninformation processing system of an embodiment of the presentdisclosure;

FIG. 6 is a diagram illustrating an exemplary processing priority in aninformation processing system of an embodiment of the presentdisclosure;

FIG. 7 is a flowchart illustrating an exemplary process in aninformation processing system of an embodiment of the presentdisclosure;

FIG. 8 is a block diagram illustrating another exemplary configurationof a cell-phone of an embodiment of the present disclosure;

FIG. 9 is a diagram illustrating an exemplary acceleration detected by asensor provided in a small-size terminal of an embodiment of the presentdisclosure;

FIG. 10 is a diagram illustrating another exemplary accelerationdetected by a sensor provided in a small-size terminal of an embodimentof the present disclosure;

FIG. 11 is a flowchart illustrating an exemplary process in aninformation processing system of an embodiment of the presentdisclosure;

FIG. 12 is a block diagram illustrating another exemplary configurationof a cell-phone of an embodiment of the present disclosure;

FIG. 13 is a diagram illustrating a threshold value being set foracceleration detected by a sensor provided in a small-size terminal ofan embodiment of the present disclosure;

FIG. 14 is a diagram illustrating a timeout period being set fordetecting a gesture; and

FIG. 15 is a diagram illustrating another exemplary informationprocessing system of an embodiment of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

Hereinafter, (a) preferred embodiment(s) of the present disclosure willbe described in detail with reference to the appended drawings. In thisspecification and the appended drawings, structural elements that havesubstantially the same function and structure are denoted with the samereference numerals, and repeated explanation of these structuralelements is omitted.

Note that description will be provided in the following order.

1. System configuration2. Exemplary process relating to an activation timing of a sensor forrecognizing a gesture operation3. An exemplary process in which a gesture operation is recognized inparallel with another operation method4. An exemplary process in which a recognized gesture operation iscanceled5. An exemplary process in which a gesture operation is recognized withmachine learning6. An exemplary process in which a different operation is recognized inaccordance with the strength of a gesture to be recognized7. An exemplary process in which a timeout period is set for recognizinga gesture

8. Supplement 9. Conclusion 1. SYSTEM CONFIGURATION <1-1. Outline ofSystem Configuration>

FIG. 1 is a diagram illustrating an outline of an information processingsystem of an embodiment of the present disclosure. The informationprocessing system of the embodiment of the present disclosure includes asmall-size terminal 100, a cell-phone 200, a communication network 300,and a server 400. The small-size terminal 100, which is a terminal to beattached to a user's ear, is connected to the cell-phone 200 viashort-distance wireless communication. The small-size terminal 100,including a loud speaker, allows the user to talk using the small-sizeterminal 100. The small-size terminal 100, also including buttons,allows the user to operate various functions or applications of thecell-phone 200 by operating the buttons. Note that the small-sizeterminal 100, the cell-phone 200, and the server 400 are exemplaryinformation processing apparatuses.

The cell-phone 200, including a loud speaker and a microphone, cancommunicate with other terminals by communicating with a base stationincluded in a mobile communication network. In addition, the cell-phone200 includes a display unit for displaying information, as well as atouch panel formed on the display unit. A user performs variousoperations relating to functions or applications of the cell-phone 200by performing operations using the touch panel. Note that the componentused for operating the cell-phone 200, which is not limited to a touchpanel, may be a component, such as a direction key, a hard button, or ajog dial.

In addition, the cell-phone 200 has various functions and applicationsbesides the call function, some of which may be realized bycommunicating with the server 400 via the communication network 300. Forexample, email, short message, social networking service, photography,music reproduction, browsing function, map display, and the like, may beincluded as types of functions and applications of the cell-phone 200.

The server 400 is connected to the cell-phone 200 via the communicationnetwork 300. The server 400 executes processes in accordance withapplications or functions of the cell-phone 200. For example, the server400 processes voice data transmitted from the cell-phone 200.

In the aforementioned system configuration, a user can operate functionsof the cell-phone 200 or applications being executed in the cell-phone200 by performing various operations on the small-size terminal 100,with the small-size terminal 100 being attached to the user's ear.

<1-2. Configuration of Small-Size Terminal>

The communication system of the embodiment of the present disclosure hasthus been outlined above. In the following, there will be described aconfiguration of the small-size terminal 100 of the embodiment of thepresent disclosure. FIG. 2 is a diagram illustrating the appearance ofthe small-size terminal 100. The small-size terminal 100, which is aterminal to be attached to a user's ear, has an insertion unit 122 to beinserted into the user's ear. In addition, the small-size terminal 100has a button 102 to be operated by the user. Note that the button 102 isan exemplary operation unit.

The appearance of the small-size terminal 100 has been described above.In the following, there will be described a configuration of theinformation processing system of the present embodiment. FIG. 3 is ablock diagram illustrating the configuration of the informationprocessing system. The small-size terminal 100 includes the button 102,a processing unit 104, and a battery 106. In addition, the small-sizeterminal 100 includes a loud speaker 108, a microphone 110, a sensor112, and a short-distance communication unit 114.

The user can perform a plurality of different operations on the button102. For example, the user may press the button 102 once briefly, ortwice briefly, as different operation methods performed by the user onthe button 102. Alternatively, the user may press the button 102 for apredetermined time period, as a different operation method performed byuser on the button 102. Note that the button 102 may be an electrostaticor a pressure-sensitive touch panel, or may be a physical key.Alternatively, there may be one or more buttons 102, the number of whichis not limited. However, a smaller number of the buttons 102 arepreferred due to the limited space for arranging the buttons 102 in asmall-size terminal, such as the small-size terminal 100 of the presentembodiment.

The processing unit 104, which is connected to each part of thesmall-size terminal 100, processes information from each part. Forexample, the processing unit 104 detects a signal resulting from theuser's operation to the button 102. In addition, the processing unit 104processes voice data received from the cell-phone 200 via theshort-distance communication unit 114, and outputs the processed voicedata to the loud speaker 108. In addition, the processing unit 104processes the voice received by the microphone 110 to generate voicedata, and transmits the voice data to the cell-phone 200 via theshort-distance communication unit 114.

In addition, the processing unit 104 processes a signal from the sensor112. The sensor 112 includes a proximity sensor 116, an accelerationsensor 118, and a gyro sensor 120. The processing unit 104 detects thatthe small-size terminal 100 is attached to the user's ear, on the basisof a detection signal generated by the proximity sensor. In addition,the processing unit 104 processes signals output from the accelerationsensor 118 and the gyro sensor 120, and transmits the processed signalto the cell-phone 200. The battery 106 supplies electric power storedtherein to each part of the small-size terminal 100, and each partoperates by the supplied power.

The loud speaker 108 outputs voice on the basis of the voice dataprocessed by the processing unit 104. When, for example, the callfunction is being used in the cell-phone 200, the loud speaker 108outputs the voice of the call on the basis of voice data transmittedfrom the cell-phone 200. In addition, the microphone 110 receives voiceand sends an audio signal to the processing unit 104. For example, themicrophone 110 receives voice uttered by the user and sends an audiosignal to the processing unit 104.

Note that, the proximity sensor 116 may be, for example, a proximitysensor capable of detecting a target object without touching the object.The proximity sensor may detect a target object using infrared light,supersonic wave, or electromagnetic wave, for example. In yet anotherexample, the proximity sensor may detect a target object using variationof eddy current due to electromagnetic induction or variation ofelectrostatic capacitance caused by approaching of the target object.

In addition, the acceleration sensor 118 detects acceleration applied tothe small-size terminal 100. When, for example, the small-size terminal100 is attached to the user, the acceleration sensor 118 detects theacceleration when the user shakes his/her neck. There are various typesof the acceleration sensor 118, such as optical or semiconductor, andthe small-size terminal 100 of the present embodiment may include anytype of acceleration sensor.

In addition, the gyro sensor 120 detects an angular velocity and anangular acceleration of the small-size terminal 100. When, for example,the small-size terminal 100 is attached to the user, the gyro sensor 120detects the angular velocity and the angular acceleration when the usershakes his/her neck. There are also various types of the gyro sensor120, such as hydraulic or optical, as with the acceleration sensor 118,and the small-size terminal 100 of the present embodiment may includeany type of gyro sensor.

In addition, the short-distance communication unit 114 uses ashort-distance wireless communication interface, such as Bluetooth(registered trademark), to communicate with other devices, such as thecell-phone 200. Since the small-size terminal 100 uses the low-capacitybattery 106, it is preferred to use a power-saving wirelesscommunication interface, such as Bluetooth Low Energy. Note that, theshort-distance communication unit 114 is not limited to theaforementioned interface and may be a short-distance wirelesscommunication interface such as ZigBee (registered trademark).

Note that, the small-size terminal 100 and the cell-phone 200 areconnected to each other by pairing. Pairing is performed by searchingfor devices located within a reachable range of radio wave and selectinga device that the user wants to connect from the detected devices. Asfor pairing, the small-size terminal 100 and the cell-phone 200 may beconfigured to authenticate each other using a password. The cell-phone200 and the small-size terminal 100 are allowed to communicate with eachother by performing pairing in the aforementioned manner.

<1-3. Configuration of Cell-Phone and Server>

The configuration of the small-size terminal 100 has been describedabove. In the following, there will be described a configuration of thecell-phone 200 of the present embodiment. The cell-phone 200 includes aprocessing unit 202, a short-distance communication unit 204, and acellular communication unit 206, as illustrated in FIG. 3. In addition,the processing unit 202 has a voice generation unit 208, a gesturerecognition unit 210, a button operation recognition unit 212, a voiceoperation recognition unit 214, and a process execution unit 216.

The short-distance communication unit 204 uses a short-distance wirelesscommunication interface to communicate with other devices, such as thesmall-size terminal 100. The short-distance communication unit 204 maybe a short-distance wireless communication interface, such as Bluetoothor ZigBee as described above. The cellular communication unit 206 isused for communicating with devices located remotely compared with theshort-distance communication unit 204. For example, the cellularcommunication unit 206 may be an interface based on a communicationstandard specified by 3GPP, such as Long Term Evolution.

The voice generation unit 208 generates various voice data. When, forexample, a call function is being executed in the cell-phone 200, thevoice generation unit 208 generates voice data of the communicationparty. In addition, the voice generation unit 208 generates voice databased on an application being executed in the cell-phone 200. Note that,the generated voice data may be sent to the small-size terminal 100 viathe short-distance communication unit 204.

The gesture recognition unit 210 recognizes the user's gesture on thebasis of the signal output by the acceleration sensor 118 and/or thegyro sensor 120. A gesture may be recognized by, for example, comparinga pattern of a signal of the acceleration sensor 118 and/or the gyrosensor 120 associated with a preliminarily stored predetermined gesturewith a signal output by the acceleration sensor 118 and/or the gyrosensor 120 and received by the gesture recognition unit 210. The gesturerecognized by the gesture recognition unit 210 may be, for example, theuser's shaking of his/her head sideways or lengthways with thesmall-size terminal 100 being attached to the user's ear. The gesturerecognition unit 210 may recognize that the user has performed anaffirmative operation by shaking his/her head lengthways. Alternatively,the gesture recognition unit 210 may recognize that the user hasperformed a negative operation by shaking his/her head sideways.

The button operation recognition unit 212 recognizes that the user hasoperated the button 102 of the small-size terminal 100. For example, theuser's operation of the button 102 may be recognized by sending of abutton operation signal from the small-size terminal 100 to thecell-phone 200 when the user operates the button 102.

The voice operation recognition unit 214 recognizes a voice operationperformed by the user. For example, the microphone 110 of the small-sizeterminal 100 receives the user's voice, and the small-size terminal 100transmits the user's voice data to the cell-phone 200. The cell-phone200 which has received the voice data transmits the voice data to theserver 400. A voice analysis unit 404 of the server 400 analyzes thereceived voice data and sends an analysis result to the cell-phone 200.The voice operation recognition unit 214 recognizes the voice operationperformed by the user on the basis of the received analysis result.

The process execution unit 216 executes various processes. For example,the process execution unit 216 executes an application installed in thecell-phone 200. In addition, the process execution unit 216 receivesoperation information from the gesture recognition unit 210, the buttonoperation recognition unit 212, and the voice operation recognition unit214, and performs a process based on the operation information, on theapplication being executed.

Note that, the analysis of voice data in the aforementioned example isperformed in the voice analysis unit 404 of the server 400. However, theanalysis of voice data performed in the voice analysis unit 404 of theserver 400 may be performed in the voice operation recognition unit 214of the cell-phone 200.

Note that, the gesture recognized by the gesture recognition unit 210may be user's tilting of his/her head to the left or right. When, forexample, the gesture recognition unit 210 recognizes that the user hastilted his/her head to the left or right, the process execution unit 216may fast forward or rewind a music to be reproduced in a musicreproduction application. Alternatively, the gesture recognized by thegesture recognition unit 210 may be the user's pushing of his/her faceforward. When, for example, the gesture recognition unit 210 recognizesthat the user has pushed his/her face forward, the process executionunit 216 may magnify a photograph in a photograph display application.Alternatively, the gesture recognized by the gesture recognition unit210 may the user's pulling of his/her face backward. When, for example,the gesture recognition unit 210 recognizes that the user has pulledhis/her face backward, the process execution unit 216 may shrink thephotograph in the photograph display application.

2. EXEMPLARY PROCESS RELATING TO AN ACTIVATION TIMING OF A SENSOR FORRECOGNIZING A GESTURE OPERATION

A system configuration of the present embodiment has been describedabove. In the following, there will be described an exemplary process inan information processing system of the present embodiment. FIG. 4 is aflowchart illustrating a timing at which the acceleration sensor 118and/or the gyro sensor 120 to be used for gesture recognition isactivated. The flowchart of FIG. 4 describes an exemplary process to beexecuted when the cell-phone 200 receives a message in an application,such as a short message service or a social networking service.

First, at S102, the cell-phone 200 receives a message. On this occasion,the application being executed by the process execution unit 216 sends anotification indicating message reception to the voice generation unit208, the gesture recognition unit 210, and the voice operationrecognition unit 214. Next, at S104, the voice generation unit 208 whichhas received the message from the application generates voice data inorder to audibly notify the user of the reception of the message atS102. The cell-phone 200 then transmits the generated voice data to thesmall-size terminal 100, and the processing unit 104 of the small-sizeterminal 100 reproduces the received voice data. Here, the content ofthe voice data to be reproduced may be, for example, “A message has beenreceived. Will it be read?”.

At S106, the voice operation recognition unit 214 which has received thenotification from the application transitions to a voice operationwaiting state to wait for a voice operation from the user. In parallelwith the processing at S106, the sensor of the small-size terminal 100is activated at S108. This may be realized by generation of anactivation signal by the gesture recognition unit 210 which has receivedthe notification from the application, and transmission of theactivation signal to the small-size terminal 100.

At S108, the acceleration sensor 118 and/or the gyro sensor 120 to beused for gesture recognition are activated. At S110, the gesturerecognition unit 210 transitions to a gesture waiting state to wait fora gesture operation from the user, in synchronization with theactivation of the acceleration sensor 118 and/or the gyro sensor 120.Note that, the activation of the acceleration sensor 118 and/or the gyrosensor 120 may be realized by sending, to the gesture recognition unit210, information indicating that the voice operation recognition unit214 has transitioned to the voice operation waiting state. On thisoccasion, the activation of the acceleration sensor 118 and/or the gyrosensor 120 synchronizes with the transition of the voice operationrecognition unit 214 to the voice operation waiting state.

Next, at S112, a sound effect is reproduced indicating that the gesturerecognition unit 210 is in the gesture waiting state. The sound effectallows the user to recognize when the gesture recognition unit 210 hastransitioned to the gesture waiting state. Next, at S114, the gesturerecognition unit 210 determines whether or not the user has performed agesture. When the gesture recognition unit 210 recognizes a gesture atS114, the process flow proceeds to S116.

At S116, operation information relating to the gesture recognized atS114 is temporarily stored in a storage unit which is not illustrated.Next, at S118, the sound effect indicating that the gesture recognitionunit 210 is in the gesture waiting state is stopped. Then, at S120, thegesture recognition unit 210 releases the gesture waiting state andterminates gesture recognition. Accordingly, the acceleration sensor 118and/or the gyro sensor 120 transition to a dormant state at S122.

Next, at S124, the process execution unit 216 executes, on theapplication, a process relating to the gesture temporarily stored atS116. Specifically, when the content of the voice data reproduced atS104 is “A message has been received. Will it be read?” and the user hasperformed a gesture of shaking his/her head lengthways at S114, theprocess execution unit 216 executes a process of reading the receivedmessage at S124.

When the gesture recognition unit 210 does not recognize a gesture atS114, the process flow proceeds to S126. At S126, the gesturerecognition unit 210 determines whether or not a predetermined timeperiod has elapsed. When the gesture recognition unit 210 determinesthat a predetermined time period has elapsed at S126, the process flowproceeds to S128.

At S128, the sound effect indicating that the gesture recognition unit210 is in the gesture waiting state is stopped. Then, at S130, thegesture recognition unit 210 releases the gesture waiting state andterminates gesture recognition. Accordingly, the acceleration sensor 118and/or the gyro sensor 120 transition to a dormant state at S132.

As has been described above, the acceleration sensor 118 and/or the gyrosensor 120 are activated on the basis of reception of a notificationindicating message reception from the application by the gesturerecognition unit 210. Activating the acceleration sensor 118 and/or thegyro sensor 120 on the basis of the notification from the application asthus described causes the sensor 112 to be activated at an appropriatetiming with regard to the operation of the application, whereby it ispossible to reduce power consumption of the small-size terminal 100.This is very advantageous for a terminal having the low-capacity battery106, such as the small-size terminal 100.

In addition to the aforementioned processing, the sensor 112 to beactivated may be selected in accordance with the notification fromapplication or the type of application. When, for example, the gestureto be recognized on the basis of the notification from application orthe type of application is a simple gesture, either the accelerationsensor 118 or the gyro sensor 120 may be activated. Alternatively, whena complicated gesture is the gesture to be recognized, both theacceleration sensor 118 and the gyro sensor 120 may be activated. Onthis occasion, the notification from application or the type thereof maybe stored in association with the gesture to be recognized in accordancewith the notification or the type.

Note that, the notification from application in the aforementionedexample is a notification indicating that a message has been received.In addition, the gesture to be recognized by the notification from theapplication is a user's gesture of shaking his/her head lengthways(affirmative response) or a user's gesture of shaking his/her headsideways (negative response).

Alternatively, a mode of the sensor 112 to be activated may be selectedon the basis of the notification from application or the type ofapplication. The mode of the sensor 112 to be selected may be, forexample, a mode having a different sampling frequency. A mode with ahigh sampling frequency is selected when a gesture with a fast motion isrecognized, whereas a mode with a low sampling frequency is selectedwhen a gesture with a slow motion is recognized. As thus described,selection of the sensor 112 or selection of the mode of the sensor 112to be activated in accordance with the gesture to be recognized allowsfurther reduction in power consumption of the small-size terminal 100.

Note that, in the aforementioned embodiment, a sound effect isreproduced at S112, which indicates that the gesture recognition unit210 is in a gesture waiting state. However, the sound effect may bereproduced while the user is performing a gesture, i.e., while theacceleration sensor 118 and/or the gyro sensor 120 are detectingacceleration, angular velocity, or angular acceleration. Alternatively,vibration may be used instead of a sound effect to indicate that thegesture recognition unit 210 is in a gesture waiting state.Alternatively, a message may be reproduced instead of a sound effect.For example, the message may be something like “Gesture recognition willbe started”. Alternatively, a message may likewise be reproduced whengesture recognition is terminated. On this occasion, a message, such as“gesture recognition will be terminated”, for example, may bereproduced.

Alternatively, when the user performs a voice operation instead of agesture operation in the aforementioned exemplary process, theacceleration sensor 118 and/or the gyro sensor 120 may transition to adormant state in synchronization with the recognition of the voiceoperation.

3. AN EXEMPLARY PROCESS IN WHICH A GESTURE OPERATION IS RECOGNIZED INPARALLEL WITH ANOTHER OPERATION METHOD

An exemplary process relating to an activation timing of the sensor 112used for gesture operation in the present embodiment has been describedabove. In the following, there will be described an exemplary process ofrecognizing a gesture operation and another operation method. In theinformation processing system of the present embodiment, recognition ofa voice operation and a button operation is performed in parallel withrecognition of a gesture operation. Accordingly, the user can performvarious operations by combining a plurality of operation methods. FIG. 5is a diagram illustrating an exemplary process in which recognition of avoice operation and a button operation is performed in parallel withrecognition of a gesture operation.

The cell-phone 200 receives a message at S202 in a similar manner to theexemplary process in FIG. 4. Next, at S204, the voice operationrecognition unit 214 transitions to a voice operation waiting state towait for a voice operation from the user, on the basis of reception of anotification indicating that a message has been received from anapplication.

In parallel with the processing at S204, the acceleration sensor 118and/or the gyro sensor 120 of the small-size terminal 100 are activatedat S206, and the gesture recognition unit 210 transitions to a gesturewaiting state to wait for a gesture operation from the user.Additionally, in parallel with the processing at S204 and S206, thebutton operation recognition unit 212 transitions to a button operationwaiting state to wait for a button operation from the user (S208).

Then, at S210, the gesture recognition unit 210 determines whether ornot the user has performed a gesture. When the gesture recognition unit210 recognizes a gesture at S210, the process flow proceeds to S212. AtS212, operation information relating to the gesture recognized at S210is temporarily stored. Note that, when the gesture recognition unit 210does not recognize a gesture at S210, the process flow proceeds to S214.At S214, the gesture recognition unit 210 determines whether or not apredetermined time period has elapsed. When the gesture recognition unit210 determines at S214 that a predetermined time period has elapsed, theprocess is terminated.

Next, at S216, the button operation recognition unit 212 determineswhether or not the user has operated the button 102. When the buttonoperation recognition unit 212 recognizes at S216 that the user hasoperated the button 102, the button operation recognition unit 212 sendsoperation information relating to a button operation to the processexecution unit 216, and the process execution unit 216 executes aprocess relating to the button operation (S218).

When the button operation recognition unit 212 does not recognize atS216 that the user has operated the button 102, the process flowproceeds to S220. At S220, the voice operation recognition unit 214determines whether or not the user has performed a voice operation. Whenthe voice operation recognition unit 214 recognizes at S220 that theuser has performed a voice operation, the voice operation recognitionunit 214 sends operation information relating to the voice operation tothe process execution unit 216, and the process execution unit 216executes a process relating to the voice operation (S222).

When the voice operation recognition unit 214 does not recognize at S220that the user has performed a voice operation, the process flow proceedsto S224. At S224, the gesture recognition unit 210 sends operationinformation relating to the gesture temporarily stored at S212 to theprocess execution unit 216, and the process execution unit 216 executesa process relating to the gesture operation.

An exemplary process when gesture operation is performed first has beendescribed above. However, the order of performing operations is notlimited to the example described in FIG. 5. FIG. 6 is a diagramillustrating a relation between an operation method performed earlierand an operation method subsequently performed.

First, there will be described a relation with another operation methodin a case where a button operation is performed earlier. When a buttonoperation is performed earlier and subsequently a gesture operation isperformed, a process relating to the button operation is executed inpreference to the gesture operation. On this occasion, a sound effectindicating that the gesture recognition unit 210 is in a gesture waitingstate may be stopped at the time point when button operation isperformed. Similarly, when a button operation is performed earlier andsubsequently a voice operation is performed, a process relating to thebutton operation is executed in preference to the voice operation.

Next, there will be described a relation with another operation methodin a case where a gesture operation is performed earlier. When a gestureoperation is performed earlier and subsequently a button operation isperformed, a process relating to the button operation is executed inpreference to the gesture operation. Similarly, when a gesture operationis performed earlier and subsequently a voice operation is performed, aprocess relating to the voice operation is executed in preference to thegesture operation.

Next, there will be described a relation with another operation methodin a case where a voice operation is performed earlier. When a voiceoperation is performed earlier and subsequently a button operation isperformed, a process relating to the button operation is executed inpreference to the voice operation. Similarly, when a voice operation isperformed earlier and subsequently a gesture operation is performed, aprocess relating to the voice operation is executed in preference to thegesture operation.

The priority of each operation method is not limited to theaforementioned example. In addition, the priority of each operationmethod may be changed. For example, weighting is performed on eachoperation method, and the process execution unit 216 may execute aprocess relating to each operation method in consideration of theweighting. The weighting of each operation method may be set by theuser. Alternatively, weighting of each operation method may be set inaccordance with the recognition precision of each operation method.

Note that, respective operation methods described above may be used incombination. For example, a button operation may be combined with agesture operation. When a music reproduction application is beingexecuted by the process execution unit 216, the user's tilting ofhis/her head to the right while touching the button 102 may allow theprocess execution unit 216 to execute a process of reproducing the nextmusic.

Alternatively, a voice operation may be combined with a gestureoperation. When a message application is being executed by the processexecution unit 216, the user's shaking of his/her head lengthways whileuttering “OK” may allow the process execution unit 216 to input theletters “OK” in an emphasized manner. In addition, the process executionunit 216 may input an image (stamp) corresponding to the letters “OK”instead of the process of emphasizing the letters.

Alternatively, a voice operation may be combined with a buttonoperation. When a message application is being executed by the processexecution unit 216, the user's uttering “OK” while touching the button102 may allow the process execution unit 216 to input the letters “OK”in an emphasized manner. Alternatively, the process execution unit 216may input an image (stamp) corresponding to the letters

“OK” instead of the process of emphasizing the letters. In theinformation processing system of the present embodiment described above,a gesture operation, a voice operation, and a button operation areperformed in parallel. Accordingly, the user can perform variousoperations using a plurality of operation methods.

4. AN EXEMPLARY PROCESS IN WHICH A RECOGNIZED GESTURE OPERATION ISCANCELED

An exemplary process in which a gesture operation, a voice operation,and a button operation are performed in parallel has been describedabove. In the following, there will be described an exemplary process ofcanceling a gesture operation. There is a possibility of misrecognizinga gesture operation and therefore it is preferred to prepare a processof canceling the recognized gesture operation. FIG. 7 is a diagramillustrating an exemplary process in which a recognized gestureoperation is canceled. In FIG. 7, there will be described an exemplaryprocess when sending back a message in a message application.

At S302, the small-size terminal 100 reproduces a voice “Reply withOK?”, for example, in order to cause the user to determine whether ornot to reply to a message. At S306, the gesture recognition unit 210transitions to a gesture waiting state to wait for a gesture operationfrom the user.

Then, at S306, the gesture recognition unit 210 determines whether ornot the user has performed a gesture. When the gesture recognition unit210 recognizes a gesture at S306, the process flow proceeds to S308.Note that, in the following, there will be described an exemplaryprocess in a case where the gesture recognized at S306 is the user'sgesture of shaking his/her head lengthways (nod).

At S308, the gesture recognition unit 210 sends operation informationrelating to the gesture recognized at S306 to the process execution unit216, and the process execution unit 216 stores the operation informationrelating to the recognized gesture. Note that, when the gesturerecognition unit 210 does not recognize a gesture at S306, the processflow proceeds to S310. At S310, the gesture recognition unit 210determines whether or not a predetermined time period has elapsed. Whenthe gesture recognition unit 210 determines at S310 that a predeterminedtime period has elapsed, the process is terminated.

Next, at S312, a voice notification is performed on the basis of thegesture recognized at S306. When the gesture recognized at S306 is agesture relating to a “nod” indicating an affirmative response, a voicenotification such as “Reply in three seconds”, for example, is performedat S312.

Next, at S314, the gesture recognition unit 210 transitions to acancellation gesture waiting state to recognize a cancellation gesturefor canceling the operation relating to the gesture recognized at S306.For example, the cancellation gesture may be the user's gesture ofshaking his/her head sideways. Here, the cancellation gesture is apredetermined specific gesture.

Next, at S316, the gesture recognition unit 210 determines whether ornot the user has performed a cancellation gesture. When the gesturerecognition unit 210 recognizes a cancellation gesture at S316, thegesture recognition unit 210 terminates gesture recognition (S318).Then, at S320, the gesture recognition unit 210 sends operationinformation relating to the cancellation gesture recognized at S316 tothe process execution unit 216, and the process execution unit 216executes a cancellation process. Specifically, the process executionunit 216 does not execute the process relating to the gesture stored atS308. In other words, the operation relating to the gesture recognizedat S306 is canceled. Then, at S322, the small-size terminal 100 performsvoice notification in order to notify the user that the operationrecognized at S306 has been canceled. The content of the voicereproduced at S322 may be, for example, “Reply canceled”.

When the gesture recognition unit 210 does not recognize a cancellationgesture at S316, the process flow proceeds to S324. At S324, the gesturerecognition unit 210 determines whether or not a predetermined timeperiod has elapsed. The time period at S324 is a cancellation period setfor the user to cancel the operation relating to the gesture recognizedat S306. Further, during the cancellation period, the process executionunit 216 waits for an input of operation information relating to thecancellation gesture from the gesture recognition unit 210.

When the gesture recognition unit 210 determines at S324 that apredetermined time period has elapsed, the process flow proceeds toS326. At S326, the gesture recognition unit 210 terminates gesturerecognition and notifies the process execution unit 216 that thecancellation period has elapsed. Then, at S328, the process executionunit 216 executes a process relating to the gesture stored at S308. AtS330, a voice notification such as “replied”, for example, is performedin order to indicate that a process relating to the gesture operationperformed by the user at S306 has been performed.

Note that, the process execution unit 216 may dynamically set the lengthof the cancellation period for waiting for an input of operationinformation relating to the cancellation gesture from the gesturerecognition unit 210. For example, the process execution unit 216 mayset the length of the cancellation period on the basis of informationacquired from an application being executed.

On this occasion, the process execution unit 216 may set the length ofthe cancellation period on the basis of the type of the applicationbeing executed. For example, the process execution unit 216 may set thelength of the cancellation period to be long for a message transmissionand reception application, and set the length of the cancellation periodto be short for a music reproduction application. This is because, intransmission of a message, the user may transmit important informationand therefore the user determines transmission of a message morecarefully. On the other hand, this is because the user suffers nosignificant disadvantage even if the user performs an erroneousoperation in music reproduction. Accordingly, there is set acancellation period of an appropriate length in accordance with the typeof the application being executed.

Alternatively, the process execution unit 216 may set the length of thecancellation period on the basis of the state of the application beingexecuted. For example, the process execution unit 216 may set the lengthof the cancellation period to be long for an operation that terminatesmoving image photographing in a moving image photographing application,and set the length of the cancellation period to be short for anoperation that starts moving image photographing. In other words, theprocess execution unit 216 sets different lengths for the cancellationperiods for an operation in a state before starting moving imagephotographing and an operation in a state during moving imagephotographing. This is because, it is generally expected to photographthe motion of the subject quickly when starting moving imagephotographing whereas, when terminating moving image photographing, itsuffices to later delete a moving image photographed for anunnecessarily long time. Accordingly, there is set a cancellation periodof an appropriate length in accordance with the state of the applicationbeing executed.

Alternatively, the process execution unit 216 may set the length of thecancellation period on the basis of the type of the notification of theapplication being executed. For example, in a message transmission andreception application, the process execution unit 216 may set the lengthof the cancellation period to be long for an operation to a notificationasking the user whether or not to transmit a message, and set the lengthof the cancellation period to be short for an operation to anotification asking the user whether or not to read a message. This isbecause the user determines transmission of a message more carefully,whereas the user suffers no significant disadvantage even if the userperforms an erroneous operation when reading a message. Accordingly,there is set a cancellation period of an appropriate length inaccordance with the type of the notification of the application beingexecuted.

Alternatively, the process execution unit 216 may set the length of thecancellation period on the basis of the magnitude of acceleration,angular velocity, or angular acceleration acquired from the accelerationsensor 118 and the gyro sensor 120. For example, when the acceleration,angular velocity, or angular acceleration acquired from the accelerationsensor 118 and the gyro sensor 120 is large, the process execution unit216 determines that user is moving and sets the length of thecancellation period to be long. This is because, the user performs manyerroneous operations when moving. Accordingly, there is set acancellation period of an appropriate length in accordance with theuser's moving state.

Alternatively, the user may set the cancellation period by operating theoperation unit of the cell-phone 200. Alternatively, the cancellationperiod may not be provided depending on type or state of application, ornotification from application. This is because it is preferred that nocancellation period is provided for an operation desired to be performedquickly. For example, it is desirable that no cancellation period isprovided for a notification “music to be reproduced?” from theapplication so that a process relating to the operation is immediatelyperformed.

Further, in FIG. 7, cancellation of a gesture operation is performed bya gesture operation. However, cancellation of an operation is notlimited thereto. For example, cancellation of an operation may beperformed by a button operation. On this occasion, the user may holddown the button 102 during the cancellation period to cause the buttonoperation recognition unit 212 to send operation information to theprocess execution unit 216, and the process execution unit 216 which hasreceived the operation information may cancel the operation.Alternatively, cancellation of an operation may be performed by a voiceoperation. On this occasion, the user may utter “no” during thecancellation period to cause the voice operation recognition unit 214 tosend operation information to the process execution unit 216, and theprocess execution unit 216 which has received the operation informationmay cancel the operation.

Alternatively, the operation information relating to the cancellationgesture at S316 may be generated when the gesture recognition unit 210detects acceleration, angular velocity, or angular acceleration equal toor higher than a predetermined value. For example, a cancellationgesture may be performed by the user's fierce shaking of his/her headsideways. Accordingly, erroneous cancellation operations may be reduced.

An exemplary process of canceling a recognized gesture operation hasbeen described above. Accordingly, in a case where a gesture operationhas been misrecognized or the user has performed an erroneous operation,the user can cancel the once-recognized gesture operation.

Alternatively, the process to be canceled in the aforementioned examplemay be other than a process relating to a gesture operation. Forexample, a process relating to a recognized voice operation may becanceled by recognition of a cancellation gesture after the voiceoperation has been recognized. Alternatively, a process relating to arecognized button operation may be canceled by recognition of acancellation gesture after the button operation has been recognized.

5. AN EXEMPLARY PROCESS IN WHICH A GESTURE OPERATION IS RECOGNIZED WITHMACHINE LEARNING

An exemplary process to cancel a recognized gesture operation has beendescribed above. In the following, there will be described an exemplaryprocess of recognizing a gesture operation using a result of learning bymachine learning. In recognition of a gesture operation, the precisionof recognition may be raised by use of a result of learning by machinelearning. However, the recognition precision of a gesture operation withmachine learning may be insufficient. Therefore, the informationprocessing system of the present embodiment performs recognition of agesture operation on the basis of a particular rule, in addition torecognition of a gesture operation by machine learning.

FIG. 8 is a block diagram illustrating another exemplary configurationof the cell-phone 200 of the present embodiment. The gesture recognitionunit 210 of the cell-phone 200 in the present embodiment has a machinelearning determination unit 218 and a rule determination unit 220. Themachine learning determination unit 218 performs recognition of agesture using a result of learning performed by machine learning. On theother hand, the rule determination unit 220 performs recognition of agesture on the basis of a predetermined rule.

FIG. 9 is a diagram illustrating a pattern of acceleration detected bythe acceleration sensor 118 when the user nods. In a range A of FIG. 9before the user nods, the acceleration detected by the accelerationsensor 118 is constant. In a range B of FIG. 9 when the user shakeshis/her head downward, the acceleration sensor 118 detects amonotonously decreasing acceleration. In a range C of FIG. 9 when theuser shakes his/her head upward, the acceleration sensor 118 detectsacceleration of an approximately same magnitude in a direction oppositeto a direction in the range B.

FIG. 10 is a diagram illustrating a pattern of acceleration detected bythe acceleration sensor 118 when the user suddenly looks down. Theacceleration pattern of FIG. 10 is similar to the acceleration patternof FIG. 9 across the ranges A to B. In the acceleration pattern in rangeC of FIG. 10, however, acceleration is not detected in the directionopposite to the direction in the range B.

The rule determination unit 220 preliminarily stores the accelerationpattern illustrated in FIG. 9, for example, as a rule of the user'snodding gesture. The rule determination unit 220 then compares thestored acceleration pattern with an acceleration pattern detected by theacceleration sensor 118 to perform recognition of a gesture. In otherwords, the rule determination unit 220 does not perform gesturerecognition because the acceleration pattern illustrated in FIG. 10 isdifferent from the stored acceleration pattern.

The aforementioned rule will be described more specifically in thefollowing. For example, acceleration before starting a gesture is set as(x0, y0, z0), and acceleration when terminating the gesture is set as(x1, y1, z1). The rule determination unit 220 may use as a rule that thevalue of difference |(x1−x0), (y1−y0), (z1−z0) between magnitudes of thetwo accelerations is equal to or smaller than a predetermined thresholdvalue.

FIG. 11 is a flowchart illustrating an exemplary gesture recognitionprocess to be executed by the gesture recognition unit 210 of thepresent embodiment. First, at S402, the machine learning determinationunit 218 performs gesture recognition on the basis of a result oflearning with machine learning. When the machine learning determinationunit 218 recognizes a gesture at S402, the process flow proceeds toS404. At S404, the rule determination unit 220 performs gesturerecognition on the basis of a predetermined rule, independently of themachine learning determination unit 218. Note that, when the machinelearning determination unit 218 does not recognize a gesture at S402,the process is terminated.

When the rule determination unit 220 recognizes a gesture at S404, theprocess flow proceeds to S406. At S406, the gesture recognition unit 210sends operation information relating to the recognized gesture to theprocess execution unit 216, and the process execution unit 216 executesa process relating to the recognized gesture. When the ruledetermination unit 220 does not recognize a gesture at S404, the processis terminated. On this occasion, the process execution unit 216 does notexecute a process relating to the gesture recognized by the machinelearning determination unit 218 at S402.

As has been described above, a gesture is recognized by the machinelearning determination unit 218 and the rule determination unit 220 inthe gesture recognition unit 210 of the present embodiment. Accordingly,misrecognition of gestures by machine learning is reduced.

6. AN EXEMPLARY PROCESS IN WHICH A DIFFERENT OPERATION IS RECOGNIZED INACCORDANCE WITH THE STRENGTH OF A GESTURE TO BE RECOGNIZED

An exemplary process in which a gesture is recognized on the basis of aresult of learning with machine learning and a predetermined rule hasbeen described above. In the following, there will be described anexemplary process in which the operation to the recognized gesturevaries in accordance with the strength of the gesture to be recognized.For example, the user may fiercely shake his/her head sideways whenexpressing strong negation, or may slowly shake his/her head sidewayswhen expressing weak negation. In the information processing system ofthe present embodiment, the user can perform different operations inaccordance with the strength of such gestures.

FIG. 12 is a block diagram illustrating a configuration of thecell-phone 200 of the present embodiment. The gesture recognition unit210 of the cell-phone 200 in the present embodiment has a strengthdetermination unit 222. The strength determination unit 222 recognizes afirst gesture when the acceleration, angular velocity, or angularacceleration detected by the acceleration sensor 118 or the gyro sensor120 is smaller than a predetermined magnitude. In addition, the strengthdetermination unit 222 recognizes a second gesture when theacceleration, angular velocity, or angular acceleration is larger than apredetermined magnitude.

FIG. 13 is a diagram illustrating a pattern of acceleration of the firstgesture and the second gesture recognized by the gesture recognitionunit 210 of the present embodiment. Here, the predetermined magnitude ofacceleration used for discriminating the aforementioned first gestureand second gesture is expressed by a threshold value T. Therefore, agesture recognized in a range A of FIG. 13 is the first gesture, and agesture recognized in a range B is the second gesture. Note that, in thefollowing, there will be described an example in which different gestureoperations are performed in accordance with the magnitude ofacceleration detected by the acceleration sensor 118.

When the user slowly shakes his/her head sideways, the accelerationsensor 118 detects acceleration of a magnitude that does not exceed thethreshold value T, as illustrated in the range A of FIG. 13. The gesturerecognition unit 210 recognizes the gesture as the first gesture, andsends operation information relating to the first gesture to the processexecution unit 216. When, alternatively, the user fiercely shakeshis/her head sideways, the acceleration sensor 118 detects accelerationof a magnitude exceeding the threshold value T as illustrated in therange B of FIG. 13. The gesture recognition unit 210 recognizes thegesture as the second gesture, and sends operation information relatingto the second gesture to the process execution unit 216.

The process execution unit 216 which has received the operationinformation relating to the first gesture and the operation informationrelating to the second gesture executes a different process for eachpiece of the operation information. For example, the process relating tothe first operation information may be a process of inputting “NO” witha usual size and color in the message input operation. On the otherhand, the process relating to the second operation information may be aprocess of inputting “NO” with a size larger than usual and red color inthe message input operation. As thus described in the aforementionedexample, the user can input a strong negative message by fiercelyshaking his/her head.

As has been described above, the user can perform different operationsby performing gestures with different strengths. Note that, differentoperations are not limited to the aforementioned example. In the callapplication, for example, the process execution unit 216 may execute a“hold-on” operation when the user has performed the first gesturedescribed above. Alternatively, the process execution unit 216 mayexecute a “hang up” operation when the user has performed the secondgesture.

Further, an example has been described above in which the user canperform the first gesture and the second gesture by performing gestureswith different strengths. However, there may be set a plurality of theaforementioned threshold values T, for example, different thresholdvalues T1, T2 and T3 in ascending order, so as to allow other differentgestures to be recognized. For example, a first gesture may berecognized when acceleration, angular velocity, or angular accelerationwhich is smaller than the threshold value T1 is detected. Alternatively,a second gesture may be recognized when acceleration, angular velocity,or angular acceleration which is larger than the threshold value T1 andsmaller than the threshold value T2 is detected. Alternatively, a thirdgesture may be recognized when acceleration, angular velocity, orangular acceleration which is larger than the threshold value T2 andsmaller than the threshold value T3 is detected. Alternatively, a fourthgesture may be recognized when acceleration, angular velocity, orangular acceleration which is larger than the threshold value T3 isdetected. As thus described, there may be set a plurality of thresholdvalues for acceleration, angular velocity, or angular acceleration to bedetected, allowing a plurality of gesture operations to be recognized inaccordance with the set threshold value.

Further, different gesture operations are recognized in accordance withthe strength of gestures in the aforementioned example. However,different gesture operations may also be recognized in accordance withthe number of times a gesture is recognized. For example, it isconceivable that the user shakes his/her head many times when expressingstrong negation. Therefore, the process execution unit 216 may executethe process relating to the second gesture when a same gesture has beenrecognized not less than a predetermined number of times. Alternatively,the process execution unit 216 may execute the process relating to thefirst gesture when a same gesture has been recognized less than apredetermined number of times. Alternatively, there may be set aplurality of threshold values with regard to the number of times agesture is detected, and a plurality of gesture operations may berecognized in accordance with the number of times of set thresholdvalue.

7. AN EXEMPLARY PROCESS IN WHICH A TIMEOUT PERIOD IS SET FOR RECOGNIZINGA GESTURE

An exemplary process in which an operation of a gesture recognized inaccordance with the strength of the recognized gesture varies has beendescribed above. Note that, in the following, there will be described anexemplary process in which a timeout period for recognizing a gesture isset. In gesture recognition, there is a case where the user shakeshis/her head sideways for a plurality of times, for example. On thisoccasion, there is a possibility that, although the user wants toperform an operation once, the operation may be performed for aplurality of times. In the present embodiment, a timeout period relatingto a gesture recognition is set in order to prevent an operation frombeing performed against the user's intention.

FIG. 14 illustrates a timeout period relating to gesture recognition tobe set in the present embodiment. Note that, in the following, therewill be described an example in which a gesture is recognized on thebasis of acceleration detected by the acceleration sensor 118. In FIG.14, the user performs a gesture such as shaking his/her head sideways,and thereby the acceleration sensor 118 detects acceleration. Thegesture recognition unit 210 starts counting of a timeout period fromthe time point of detecting the acceleration.

A first-time gesture is then recognized at a time point denoted by P1.Subsequently, a change of acceleration caused by a plurality of times ofsimilar gestures performed by the user is detected. On this occasion,the gesture recognition unit 210 determines whether or not the timeoutperiod has elapsed. When the gesture recognition unit 210 determinesthat the timeout period has not elapsed, the gesture recognition unit210 does not recognize gestures performed during the period. Then, afterthe timeout period has elapsed, the gesture recognition unit 210 startsrecognizing a gesture. Accordingly, the gesture recognition unit 210recognizes a second-time gesture at a time point P2 after the timeoutperiod has elapsed.

As has been described above, setting a timeout period prevents a processfrom being performed for a number of times unwanted by the user.

8. SUPPLEMENT

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

In the aforementioned embodiment, for example, recognition of gestures,recognition of button operations, and recognition of voice operationsare performed by the cell-phone 200. However, recognition of gestures,recognition of button operations, and recognition of voice operationsmay be performed by the server 400. Specifically, the small-sizeterminal 100 transmits, to the server 400 via a network, signals fromthe sensor 112, audio signals from the microphone 110, and operationsignals of the button 102. The server 400 which has received respectivesignals processes the received signals to perform recognition ofgestures, recognition of button operations, and recognition of voiceoperations. The server 400 then transmits the result of recognition tothe small-size terminal 100. The small-size terminal 100 then executes aprocess to an application in accordance with the received recognitionresult.

Alternatively, recognition of gestures, recognition of buttonoperations, and recognition of voice operations may be performed by thesmall-size terminal 100. On this occasion, the processing unit 104 ofthe small-size terminal 100 processes the signals from respective partsto perform recognition of gestures, recognition of button operations,and recognition of voice operations. The small-size terminal 100 thenexecutes a process to an application in accordance with the receivedrecognition result. The small-size terminal 100 may be configured toexecute all the processes as described above. Note that, in theaforementioned example, the small-size terminal 100 may be directlyconnected to the server 400 via the communication network 300 asillustrated in FIG. 15. Alternatively, the small-size terminal 100 mayperform gesture recognition and transmit operation information relatingto the recognized gesture to the cell-phone 200.

Alternatively, the small-size terminal 100 may calculate a likelihood ofgesture recognition and transmit the likelihood to the cell-phone 200,so that the cell-phone 200 may determine the gesture recognition usingthe likelihood of the gesture recognition calculated by the small-sizeterminal 100 and a predetermined threshold value. In this example, thesmall-size terminal 100 and the cell-phone 200 may be connected througha communication path such as Bluetooth, Wi-Fi, or a wired network.

The small-size terminal 100 executes a process relating to gesturerecognition on the basis of information from the sensor 112, andgenerates an intermediate result of gesture recognition. When N types ofgesture recognition are performed, an exemplary intermediate result maybe a data set having listed therein a compatible value (threshold value)for each gesture. For example, when two types of gestures, namely “nod”and “sideways shake” are recognized, the intermediate result may beexpressed as “nod:0.8”, and “sideways shake:0.2”, for example. Theintermediate result, which is a pair of a gesture and score generated inthe small-size terminal 100, is transmitted to the cell-phone 200. Thecell-phone 200 may determine the threshold value in accordance with thesetting value, personalized policy for each user, or the situation ofthe user, the small-size terminal 100, or the cell-phone 200, anddetermine whether or not to process an operation relating to gesturerecognition.

Further, in the aforementioned embodiment, the small-size terminal 100is an information processing apparatus attached to the user's ear.However, the small-size terminal 100 is not limited thereto. Forexample, the small-size terminal 100 may be a wristband-type wearableterminal.

In addition, there may be provided a computer program for operating theprocessing units 104 and 202 in the manner described above.Alternatively, there may be provided a storage medium having such aprogram stored therein.

9. CONCLUSION

As has been discussed above, the sensor 112 used for gesture recognitionin the information processing system according to the embodiment of thepresent disclosure is activated on the basis of a notification from anapplication. In addition, the sensor 112 used for gesture recognition isactivated in synchronization with a voice operation. As a result, it ispossible to reduce power consumption in the small-size terminal 100.

Additionally, the information processing system according to theembodiment of the present disclosure can wait for operations relating togesture recognition, voice operations, and button operations inparallel. In addition, the user can perform operations combiningoperations relating to gesture recognition, voice operations, and buttonoperations. Therefore, the user can perform various operations.

In addition, the information processing system according to theembodiment of the present disclosure executes a process of cancelling anoperation relating to a once-recognized gesture. Accordingly, the user,even when having performed an erroneous operation, can cancel theoperation. In addition, there is dynamically set a cancellation periodfor accepting a process of canceling an operation relating to arecognized gesture in the information processing system according to theembodiment of the present disclosure. Accordingly, there is set anappropriate cancellation period in accordance with the state or type ofthe application being executed.

Further, the effects described in this specification are merelyillustrative or exemplified effects, and are not limitative. That is,with or in the place of the above effects, the technology according tothe present disclosure may achieve other effects that are clear to thoseskilled in the art from the description of this specification.

Additionally, the present technology may also be configured as below.

(1) An information processing apparatus including:

a process execution unit configured to execute a process relating to auser's gesture recognized on a basis of information from a sensor,wherein

the process execution unit determines, during a period after the gesturehas been recognized, whether or not there is an input of particularoperation information based on a user's operation, and

the process execution unit refrains from executing the process relatingto the recognized gesture when there is an input of the operationinformation during the period.

(2) The information processing apparatus according to (1), wherein

the process execution unit dynamically sets a length of the period.

(3) The information processing apparatus according to (2), wherein

the process execution unit executes an application,

the process execution unit executes a process relating to the gesture asan operation to the application, and

the process execution unit sets the period on a basis of informationrelating to the application acquired from the application.

(4) The information processing apparatus according to (3), wherein

the process execution unit sets the period in accordance with a type ofthe application.

(5) The information processing apparatus according to (3), wherein

the process execution unit sets the period in accordance with a state ofthe application.

(6) The information processing apparatus according to (3), wherein

the process execution unit sets the period in accordance with a type ofnotification of the application.

(7) The information processing apparatus according to (2), wherein

the sensor is an acceleration sensor or a gyro sensor, and

the process execution unit sets the period in accordance with amagnitude of acceleration or angular velocity acquired from the sensor.

(8) The information processing apparatus according to any one of (1) to(7), including:

a gesture recognition unit configured to recognize the gesture, wherein

the operation information is generated on a basis of a gesturerecognized by the gesture recognition unit.

(9) The information processing apparatus according to any one of (1) to(8), including:

a gesture recognition unit configured to recognize the gesture, wherein

the sensor is an acceleration sensor or a gyro sensor, and

the operation information during the period is generated when thegesture recognition unit detects acceleration or angular velocity whichis equal to or higher than a predetermined value.

(10) The information processing apparatus according to any one of (1) to(9), wherein

the process execution unit executes an application, and

the sensor is activated on a basis of a notification from theapplication being executed by the process execution unit.

(11) The information processing apparatus according to any one of (1) to(10), wherein

a mode of a sensor to be activated is selected in accordance with anotification from the application or a type of the application.

(12) The information processing apparatus according to any one of (1) to(10), wherein

the sensor includes a plurality of sensors, and

a sensor to be activated is selected in accordance with a notificationfrom the application or a type of the application.

(13) The information processing apparatus according to any one of (1) to(12), including:

a gesture recognition unit configured to recognize the gesture, wherein

the sensor is an acceleration sensor or a gyro sensor, and

the gesture recognition unit recognizes a first gesture whenacceleration or angular velocity which is smaller than a predeterminedvalue is detected, and recognizes a second gesture when acceleration orangular velocity which is larger than the predetermined value isdetected.

(14) The information processing apparatus according to any one of (1) to(13), including:

a gesture recognition unit configured to recognize the gesture, wherein

the gesture recognition unit recognizes a first gesture when a samegesture is recognized a smaller number of times than a predeterminednumber of times, and recognizes a second gesture when a same gesture isrecognized a larger number of times than the predetermined number oftimes.

(15) The information processing apparatus according to any one of (1) to(14), including:

a gesture recognition unit configured to recognize the gesture, wherein

the gesture recognition unit includes

-   -   a machine learning determination unit configured to recognize        the gesture on a basis of machine learning, and    -   a rule determination unit configured to recognize the gesture on        a basis of a predetermined rule, and

the gesture recognition unit recognizes the gesture when both themachine learning determination unit and the rule determination unitrecognize the gesture.

(16) An information processing system including:

a process execution unit configured to execute a process relating to auser's gesture recognized on a basis of information from a sensor,wherein

the process execution unit determines, during a period after the gesturehas been recognized, whether or not there is an input of operationinformation based on a user's operation, and

the process execution unit refrains from executing the process relatingto the recognized gesture when there is an input of the operationinformation during the period.

(17) An information processing method including:

executing a process relating to a user's gesture recognized on a basisof information from a sensor;

determining, during a period after the gesture has been recognized,whether or not there is an input of operation information based on auser's operation; and

refraining from executing the process relating to the recognized gesturewhen there is an input of the operation information during the period.

What is claimed is:
 1. An information processing apparatus comprising:processing circuitry configured to execute an application; identify aprocess to be executed in the application based on a user's gesturerecognized on a basis of information from a sensor; determine whether ornot a period is provided on the basis of information relating to theapplication acquired from the application; in a case that the period isdetermined to be provided, determine, during the period after thegesture has been completed and recognized, whether or not there is aninput of particular operation information based on a user's operation,wherein the particular operation information is at least one of a usergesture, a voice input, or a button selection; and refrain fromexecuting the process relating to the recognized gesture when there isan input of the operation information during the period.
 2. Theinformation processing apparatus according to claim 1, wherein theprocessing circuitry is configured to: dynamically set a length of theperiod to a first length in a case that the application is anapplication program of a first type; and dynamically set the length ofthe period to a second length, which is different from the first length,in a case that the application is an application program of a secondtype, wherein the first type of application program is different thanthe second type of application program.
 3. The information processingapparatus according to claim 1, wherein the processing circuitry isconfigured to dynamically set a length of the period.
 4. The informationprocessing apparatus according to claim 1, wherein the processingcircuitry is configured to execute a process relating to the gesture inthe application; and set the length of the period on a basis ofinformation relating to the application acquired from the application.5. The information processing apparatus according to claim 4, whereinthe processing circuitry is configured to set the period in accordancewith a state of the application.
 6. The information processing apparatusaccording to claim 4, wherein the processing circuitry is configured toset the period in accordance with a type of notification of theapplication.
 7. The information processing apparatus according to claim1, wherein the sensor is an acceleration sensor or a gyro sensor, andthe processing circuitry is configured to set the period in accordancewith a magnitude of acceleration or angular velocity acquired from thesensor.
 8. The information processing apparatus according to claim 1,wherein the processing circuitry is configured to: recognize a secondgesture; and generate the operation information on a basis of therecognized second gesture.
 9. The information processing apparatusaccording to claim 8, wherein the sensor is an acceleration sensor or agyro sensor, and the processing circuitry is configured to recognize athird gesture; and generate the operation information during the periodwhen the circuitry detects acceleration or angular velocity which isequal to or higher than a predetermined value based on an output of theacceleration sensor or the gyro sensor.
 10. The information processingapparatus according to claim 1, wherein the sensor is activated on abasis of a notification from the application being executed by theprocessing circuitry.
 11. The information processing apparatus accordingto claim 10, wherein a mode of a sensor to be activated is selected inaccordance with a notification from the application or a type of theapplication.
 12. The information processing apparatus according to claim10, wherein the sensor includes a plurality of sensors, and a sensor tobe activated is selected in accordance with a notification from theapplication or a type of the application.
 13. The information processingapparatus according to claim 1, wherein the sensor is an accelerationsensor or a gyro sensor, and the processing circuitry is configured torecognize a first gesture when acceleration or angular velocity which issmaller than a predetermined value is detected; and recognize a secondgesture when acceleration or angular velocity which is larger than thepredetermined value is detected.
 14. The information processingapparatus according to claim 1, wherein the processing circuitry isconfigured to: recognize a first gesture when a same gesture isrecognized a smaller number of times than a predetermined number oftimes; and recognize a second gesture when a same gesture is recognizeda larger number of times than the predetermined number of times.
 15. Theinformation processing apparatus according to claim 1, wherein theprocessing circuitry is configured to: recognize the gesture on a basisof machine learning and-on a basis of a predetermined rule.
 16. Theinformation processing apparatus according to claim 1, wherein theparticular operation is a non-gesture operation performed by the user.17. The information processing apparatus according to claim 1, whereinthe particular operation is a voice command uttered by the user.
 18. Theinformation processing apparatus according to claim 1, wherein theprocessing circuitry is configured to: receive the voice input as avoice command uttered by the user during the period after the gesturehas been recognized; determine whether the voice command corresponds toa command to cancel the process to be executed in the application; andcancel the process to be executed by the application in a case that thevoice command corresponds to a command to cancel the process to beexecuted in the application.
 19. An information processing systemcomprising: processing circuitry configured to execute an application;identify a process to be executed in the application based on a user'sgesture recognized on a basis of information from a sensor; determinewhether or not a period is provided on the basis of information relatingto the application acquired from the application; in a case that theperiod is determined to be provided, determine, during the period afterthe gesture has been completed and recognized, whether or not there isan input of particular operation information based on a user'soperation, wherein the particular operation information is at least oneof a user gesture, a voice input, or a button selection; and refrainfrom executing the process relating to the recognized gesture when thereis an input of the operation information during the period.
 20. Aninformation processing method comprising: executing an application;identifying a process to be executed in the application based on auser's gesture recognized on a basis of information from a sensor;determining whether or not a period is provided on the basis ofinformation relating to the application acquired from the application;in a case that the period is determined to be provided, determining,during the period after the gesture has been completed and recognized,whether or not there is an input of particular operation informationbased on a user's operation, wherein the particular operationinformation is at least one of a user gesture, a voice input, or abutton selection; and refraining from executing the process relating tothe recognized gesture when there is an input of the operationinformation during the period.